Steepest descent on factor graphs

x f(x, θ) log f(x, θ) exists for all θ and θ. In principle, one can apply the sum-product algorithm in order to find (1), which involves the following two steps [2]: 1. Determine f(θ) by sum-product message passing. 2. Maximization step: compute θmax △ = argmaxθ f(θ). This procedure is often not feasible, since • When the variable x is continuous, the sum-product rule may lead to intractable integrals, i.e., the (exact) computation of f(θ) is in this case intractable. • The maximization step may be infeasible. Expectation maximization is one approach to deal with both issues; the expectation maximization algorithm attempts to find (1) as follows [1] (see also [5] and [7]): 1. Make some initial guess θ. 2. Expectation step: evaluate f (θ) △ = ∫